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A brief introduction to use of projective coordinates for 
hand-eye position computations. Some standard theorems. 
Appendix A reproduces parts of Roberts' thesis concerning 
homogeneous coordinates and matching of perspactively transformed 
objects. Appendix B by Arnold Griffith derives the stereo 
calibration formulae using just the invarlance of cross-ratios 
on projections of lines , and he describes a program that uses this. 



Perspective and stereo calculations 

1.0 Redundant Coordinates 

We are used to using linearly independent, e.g. r Cartesian coordinates for 
analytic geometry. As you will see. there are sometimes advantages in using 
"redundant" coordinate systems, with an "extra" axis, because computations 
become simpler- Also, they can help with "numerical stability*" in avoiding 
divisions by keeping numerators and denominators separate until the end of 

the computation. 

■ 

1.1 Barvcentric Coordinates 

A well-known coordinate system is the barvcentric system. Choose 3 
non-collinear points in the plane 
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Then the bar vcen tri c coordinates of an arbitrary fourth point P is the 
ordered set of weights of mass-points at the P 's that would have their 
center-of-gravity at P. Of course, this is not unique* because if 
<m 1 t m*, m \) represents P, then so does (km , knu* km.)- We could set 
m- + bu + m* - i t or use the ratios m./nu, rn^/nu as unique representors* 

Given P. - (x., y ), P^ » (x^, y^> , P - (x,* y~) in Cartesian coordinates, 
we find the (normalized) barycentric coordinates of an unknown P * (x, y) by 
solving 
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so if D is the determinant then 
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There is one coordinate system in which this comes out neatly: 
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Figure 1.1 

There the barycentric coordinates of a point (y, y) are (x, y, 1 * x • y) so 
that the first two components are the Cartesian coordinates, if the multiplier 
is chosen to make the three sum to unity. 
L>2 Projective Coordinates 

The trouble with using barycentric coordinates for Vision is that they 
aren't suitably invariant under perspective changes. But this can be fixed. 1 



Choose a fourth point P and let (n » n^* n ) be its barycentric coordinat 



es 



for P , P and P . Sow define the projective coordinates of a point P with 
respect to P^ P y , P Q and t^ y to be 



(p , , r) . Q., X S) 



Now P , P , and P- are any non-collinear triple, 
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whcre (m , nu , m ) are the barycentric coordinates of P with respect to 
P » P * P . This is invariant, under a projection of any quadrilaterial 
P , P (t P., P ( into another P', P', P' r P' ( J Note that in projective 
coordinates the points 



xy 



always have components 
(1,0,0) 



(0,1,0) (0,0,-1) (l»l,l). 



For our standard system we will use a unit square. 




Figure 1.2 



In this system the projective coordinates of a Cartesian point P ■ (x, y) are 
proportional to 



(x, y, x + y - 1) 
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Here are some coordinates of point** some "normalized to make their 
- unity. 
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With a rtev quadrilaterial one see more clearly what is happening: 
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1.3 Transforming Back 

To transform back from projective to Cartesian coordinates, one assumes 
that one has a projective point (p, q> r ) -(kx» ky» k(x + y) - 1) hence 



p*q-r 



and 



p-Kj-r 



and this will work except when (p+q-r) is dangerously small, numerically. 
This happens for very large x's and/or y'$ and means that the point cannot 
be in the "table-plane" or it would be too far away! The "bad points" lie 
near the "line at infinity" which has the equation ^p + n q + n.r - 0» 
and we will discuss it later. 
2.0 Transfor ming from Eye coordinates to Table coordinates: no z -axis 

If we think of the table as a big square, whose corners are the vertices 
of Figure 1.2, its projection on the retina will look something like this: 
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and «e want to find the point in table-coordinates corresponding to an 
arbitrary point seen on the retina. (We assume the point is projected from 
the table—we'll worry about the z-axis later*) 
We need some notation* Let 

(u , \\) be retinal (!*•-* "vidissector") coordinates ; 
(x t » y L ) be table Cartesian coordinates; 
(p, q> r) be projective coordinates. 

We don't have to say, for projective coordinates, whether they are table or 
retinal , because they're invariant; How the four P-points (the table corners) 
have coordinates 
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Suppose that their retinal coordinates have in fact been detetnined—by 
experiment! — using, say, the program de veloped by Arnold Griffith, or by 
the Vision System, 

Now we can take an arbitrary point (u, v) on the retina and transform it 
to projective coordinates, as follows. 



First--once and for all--we find the barycentric coordinates of P with 

xy 

respect to P » P t > P. by using the formulae (see (1*1): 
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and these are saved for future use. How the projective coordinates (p ( q, r) 
for the point (u, v) are given by (see §1,2): 
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Observe, for programming speed) that these can be expanded as 



»[ 



'i"') l, , r Vi L r V 3 - V3 



]-["M^ 



q * etc, 
r • etc. 



where the bracketed quantities can be precomputed once and for all. 1 / 

Each retinal -* projective conversion requires only two multiplications 
and two additions per component. One doesn't have to worry about numerical 
stability at this point because the n's will be large enough. (That is why we 
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should use the whole table: so that P^ can be maximally far from the other 
P's and thus have large barycentric coordinate*,) 

Finally one has to Transform Back to Cartesian table coordinates (see 
§1.3). This need not be done immediately. We should watch this carefully, 
in fact, because often it need never be done! 

3.0 Some Theorems 

3.1 Quadrangles 

You might wonder *hy a plane quadrangle i* sufficient, or why a triangle 
is insufficient, Tne triangle isn't enough because there are 9 parameters in 
the transformation, with one of them redundant in the ratios, leaving 8. 
Various degree-of -freedom analyses will show this. You might prefer the 
following constrictive argument to the usual kind of axiomatic proofs found 
in projective-geometry books. (By the way, I am a novice about the theory; 
this Is just my initial way of understanding it, and it may bt "immature. 11 ) 
We will accept that projection of a picture through a point (the lens center 
approximation) onto another plane carries lines into lines, because pairs of 
planes intersect in lines* 

Now, given an arbitrary quadrilateral, observe that one can geometrically 
construct the "projective" midpoint of any side 
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by first finding its center S. by intersecting diagonals, then finding the 
vertex S^ by intersecting opposite sides, and finally finding the "sidpoLat" 
of P-» P by intersecting with the line through S. and S ? . It is clear that 
this is the invariant point with projective coordinates (0,-1,1). Nov 
observe that we have a new, known, quadrilateral P ft , P , S, , S- so we can do 
it again, invariantly bisecting S~, P^ or, similarly, S-, P * Repeating 
this indefinitely often, we thus can construct all binary fractions along the 
line P Q1 P y l 

Thus we obtain, by "closure/ 1 or continuity, or Dedekind Cuts. . .whatever 
you prefer, a real coordinate system along[P*, P"]like the picture in §2 
where we see the projection of a Cartesian coordinate system* 

We have thus shown, by arguments using only intersections of lines, that 
this image is uniquely constructable from the vertices of our arbitrary 
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quadrilaterial"hence it must be "protectively invariant." 
3.2 Usef u ltheorerns about homogeneous coordinates 

In addition to the Cartesian coordinates (x, y) we are interested in 
homogeneous coordinates, defined as the equivalence-classes of triples under 
multiplication of each component by the same factor k. The types we will treat 



are 



H 



(Hoooscalar) (x, y> 1) 
(Barycentric) (x> y, 1-x-y) 
(Projective) (x» y, x+y-1). 
Scale Change 

In all systens if 

(** y) = (u f v» w) 
than 

(kx, ky) s (ll> Vf E). 
Transforming Back to Cartesian 



H 



(Ui v t w) - - (u,v) 



<u, v, w) - 

(u» v, w) - 1 



1 



u+v+w 

1 

U+V-W 



<u t V), 



Lines 

We will represent a line as a column vector [u> v, wj. The Cartesian 
line ax + by + c * becomes 

H: (a, b, c) 

B; (a+c, Mc, c) 

P; (a+c, bfc. -c). 
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(The simplicity of the form for H reconxiends lc highly.) 

The point fu, v, w) is on the Line [p, q t r) if (u, v, w)*[p, q, r] - 0, 
that 1*, if up + vq + wr = 0. The sign of the scalar product tells on which 
side of the line is the point, and even its distance away, provided the vectori 
are normalized to standard form. For example, in B: 



(u, v, 1-u-v) [pfr, q+r, rj - up + vq + r. 



The line through two points (u. , v . w ) and (u., v ? , w ) is given by 



u v l u 2 



VV 1 V 2 



*1 w 2 



that Li , 

U * V 1 W 2 * V 2 V 1* + v ^ w i v 2 " W 2 U 1* + w ' u l v 2 * U 2 V 1* " ° 

because the determinant will vanish only when the first column equals the 
second, or the third, or a linear combination of thctn. 
Two lines [p^ q^ rj and [p 2> q 2 , r 2 I 



intersect in the point 
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( V2 " Vl' r l P l ■ W P i q 2 " P 2 q l> 



Useful fact: the three points 

(0,1,1) <1 ( 0,1) (iil.O) 

arc never collinear, (This Is known as Fano's Axiom.) 
Transformations 

To translate P * (x,y,v) to the origin, i.e. t to transform 
(x + a, y + p) - (a,p) apply the aatrix: P* - PT. 

»(» 4 * 

\ -x -y vj 
This works in any syscea, provided w is normally defined. 

The line through a point P normal to the line L is given by 



T(P> • K * L 



where T(P) is as above and K is 
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The line parallel to L through P is given by T(P)K L where 

/-, 0' 
K - I -1 
\0 
4,0 Finding where the eye is 

There are any number of ways to do this and the choice depends on 
(1) convenience of a "calibration object" and (2) avoidance of numerical 
degeneracy --loss of precision. The larger the calibration object, the better 
--Gosper's use of a large imaginary object whose vertices arc traced out by the 
arm allows larger spans than would be convenient for real objects in the labora* 
tory, as well as providing the important direct hand-eye-relation coefficients. 
Also, this dynamic calibration-object could be programmed to provide favorable 
positions as a function of the observations, to avoid degeneracy for a wider 
class of hand-eye position relations. 
6. 1 A Reference Cube 

We will analyze a particularly straightforward test-object--six of the 
vertices of a cube. These form two squaresat right angles. 



p--<*>,V*> 
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We will find the point F of the lens center by observing the 
projections <a A , a^ and <p Q , P^of the points P and P ,in the xz plane. 

The analysis is simple when we use the projective coordinates in both 
planes. First* observ* the geometry o£ the projections along the x and z 
axes, for a point (x, y) in the xy plane: we want to find its projection 
<x'. z') in the xz plane. 
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By similar triangles we have 



yf yf + xf 

i ' z , x v 

z y + £ * y + f 

7 y y 



For the two points ? y - <0,l> xy - (o^ a^nd P xy - UrD^ - (Pq'Vxz " 
then obtain 
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Now a , a, i P n » P, are computed as the normalized projective coordinates 
o£ the images of P and P^ in the quadrangle F Q , P^, P^, P x so we can 
assume that they are available. But then we obtain, simply: 



f * -L^o 



* " < a i " V A 



f z ■ L *G 



l - (a L - a Q > 
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Two points should be noticed: 

(1) a A and p- should be equal. Any deviation Is a measurement error: 
if Urge, the observation is suspect. If small^one probably should use 



a o + "o 



(2) The quantity L ■ -* j- — r i3 critical for accuracy. If the 

eye is relatively far from the scene, the projection will be nearly parallel; 
the a. - a« * 1 and L will be small, killing precision—as one would expect. 
For a large 2-foot object with the Eye 6 feet away* we would have L e 1/3 so 
that the precision of the F measurements is 1/3 that of the plane measurements, 
which is fine. 
4.2 Using a Matrix 

The geometric argument above is perfectly valid because the first two 
components of die projective coordinates are equal to the appropriate 
Cartesian coordinates. Let's just compute the xy - xz projective transforma- 
tion for fun. Given (x, y. x + y - 1) we want to get (x r , z' * x' + z' - 1) 
Since, 



yf + xf yf 

* * and 2 ' - — -V 

y + s y y + f y 



ve have 



yf + yf + xf + f • y 
S i + B . . i - _5 *__^ 1 

' + ', 



and the required triplet is obtained by 
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* +£ y 




f +f +f -1 -f 

x y z y 




Substituting P = (0,1,0) and P = (1,1,1) we have 



<W~> s lir<W* 



«V<V~> a iA; <W £ «^ 



which leads, of course, to the same solution as in $4.1* There is no particular 

reason, here, to use the matrix foraulation: there vould, if one used mote 

complicated coordinates, e.g., tetrahedra. The term f + f + f - 1 suggests 

that one might find value in using a spatial projective system In which the 

lens-point (f , f , f ) is a reference vertex, and Lennon is said to be 

writing a memo about this- The barycenters of F in the P fl P P P tttra- 

hedra are <f , f , f , 1 * f - f - f )- 
x y z x y z J 

To summarize the procedure to find where the eye is: 

(1) Heasure the six points positions. 

(2) Find the barycentrlc coordinates of P with respect to 

xz 

(3) Express P , P in the x* projective system, §1-2, 

(4) Use the formulae in §4.1 to find f f and f ■ 

x y z 
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5.0 Finding a space point, usinfi Stereo 

Suppose that two points (je> y) and (x 1 , y 1 ) are found in the two eyes, 
and are supposed to be images of the same point (x ,y ,z )■ Then they must 
lie on the intersection of the two lines defined by (see diagram on p. 20) 



(x. y, 0) - <f xt f y . f a > 
and 

<*'. y', 0) -<f x , f yl f z ). 

- 
If we parametrize the linos by * t £ 1 and eliminate z we obtain from the 

x equation 



E ' t - f £' - f'T 



which is the fractional distance of the intersection from the table to the 
first eye. Then the intersection is 



«l-t)x + tf x , <l-t)y+ tf y , tf^) - (at , y , z ). 



Of course, one might encounter an error because the two lines don t 
intersect; To check this, one taight also compute the other version of t* 
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t - 



1 - f £' - f'f 
y z _ y 2 



and If it doesn't agree closely something is wrong. I hope someone else will 
do a oore thoughtful error analysis! 

It would be valuable for someone co analyze the general precision question 
about all of this. Consider the following questions: 

(1) Do I need six points? Clearly not, in principle. But what about 
practice? Can we do just as well with 5 or even 4 points? 

(2) To look at it another way, if we have redundant information* can we 
use it better? We used just two rays in our analysis, but we could also use 
the projections of P in the xy plane, P in the P , P ( P ( P , etc. 

So one could use all six available rays and average, or something, to reduce 
errors. ("Something" might just be selection of the best pair.) 
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Appendix B» by Arnold Griffith 
General Approach 

A calibration object consisting of two parallel squares of edge length 
and separation s y defines a coordinate system (x, y» z) in the real world: 




A point P in space gives rise to two rays B and R_; both Incident with P, 
and respectively incident with the "points of view" of the cameras in the 
positions designated one and two: 
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L« (x.» y.) and <x , y!) be the intersections of R. with the XY plane and 
the XY' planes respectively , The basis of the stereo routine her* described 



The XY' plane may be seen from the diagram to be parallel to and with 
iintlar orientation as the XY plane* but displaced a distance s. 
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is the determination of these points. Once these are determined, the location 

of P in space follows easily. 

The Determination of an Intersection Point, an Example 

The image on the retina, i.e., the focal plane, of the square which 

defines the X¥ plane is a quadrilaterial AECD. 
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Note that nust be the image of the center of the square A'B'C'D 1 , and 
hence M and N are the images of the edge midpoint* M' and K'. Let Q be the 
image of some point in space. Since Q is the image of any point along R.> 
it is the icage of (x.> y.). It is easy to see that Y and X on the diagram 



are the images of the points (0,y.) and (x-,0): 
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Consider the computation of, e.g, , y , By a theorem of projective geometry 
v« have that 



A'X'-Y'B' 

Y'A^K'B 1 



AH'YB 
YA.J^B 



(1) 



where, e-g-> AM stands for an (unsigned) distance between A and M. It ia 
easy to see that this equation ±s still valid when AM is interpreted as a 
vector from A to M> and * is dot-product. Hence Eq. (1) becomes, eventually; 



:ia^Y' 



1 MB-AY+MA-BY . (2) 

The Determination of Position 

The rays R. and IL may not intersect because they came from two points 
which are not the same, or because of measurement errors. It is important 
to have a criterion for closeness such that rays satisfying the criterion are 
to be considered to be from the same object. An easily computed criterion 
is the difference in z-values of two points, one on each line* which have the 
same x and y coordinates- The computation involves an intersection of two 
lines in 2-space. 
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If the lines arc sufficiently close, i.e., the z-value* at (x, y) on the two 
lines differ by a sufficiently small anount, then the z value for the point 
is the average of the two z values on the lines. 

Availability ; The following programs exist and will be described in further 
detail in a forthcoming Vision Memo. 
(CAL) Does the calibration? i.e., looks at the real world and finds 

A, B, C» D» 0, P Jf ? 2 , M and K for both eyes, 
(WHERE1 P C) P is a point, C is one or two* depending on which camera position 

P is seen at. Returns ((x^YgX^y^)), 
(WHERE2 X Y) X is Che value of (WHERE1 P 1) , y is (WHERE1 P2) . Value is 
((»> y» z) 6), where (x, y» 2) is the position of the point, P, 

and d is the z-value disparity described above. 



